building a consistent illustrated world with ai

the problem

off-the-shelf image models were good at producing individual illustrations, but bad at remembering what came before.

characters changed faces. people aged too quickly. hands broke. clothing drifted. text appeared where it should not. compositions slowly became repetitive.

the challenge became:

how do you turn an unpredictable image model into something you can actually art-direct?

rather than relying on increasingly complicated prompts, i built the pipeline around three things: style, identity and control.

the workspace before the content

the least visible phase was the largest, and it is the reason one person could keep hundreds of interdependent cards coherent at all. before a single card existed, i built the content workspace itself — nine sequential tasks, each shipped as its own commit, toward one stated goal:

hallucinated fields and flags should be impossible to ship.

the order was deliberate. the parameter registry came first, then the layered context map, then the card schema and template, then the validator — written test-first, with failing tests before a line of implementation — then the dependency graph, a pilot block, and only then something playable.

the validator is the part that changed how the project felt to work in. it is not a spell-checker: it loads the registry, walks every card, and has to exit clean before anything is committed — then regenerates the card index itself, so the index cannot drift from reality.

the same instinct produced my favourite test in the repository. the prototype and the shipped app run two separate implementations of the card-selection maths, so a test extracts both, runs them over randomised game states and asserts more than 10,000 comparisons with zero mismatches — eligibility, weights, seeded draws and applied effects all identical. the two builds cannot drift into playing two different games.

teaching the model the visual language

i began by building and curating a visual reference set.

68 candidates became 65 after cleanup, then the strongest 30 were selected for training. the final dataset deliberately mixed interiors, landscapes, night scenes, winter and daylight so the model would learn the rendering language rather than memorize a narrow family of subjects.

i used it to train a custom lora on krea-2-raw.

lora is a parameter-efficient adaptation technique: rather than retraining an entire model, a relatively small set of additional weights learns the desired adaptation.[1] that made it a practical way to carry a dedicated visual language through a large production pipeline.

one trained model was not enough to trust. i later trained two more versions with adjusted dataset balance, captioning and training strength, then ran a/b comparisons on the same fixed set of test prompts — same scenes, same seeds, only the model changed.

judging them side by side made the differences obvious in a way single-model review never did, and the winning version was the one we kept for the full production run.

keeping the same people alive for sixty years

style consistency was only half the problem. the same character could still become a different person between two scenes.

i built a small character bible separating features that had to remain recognizable from those allowed to evolve.

face structure, the eye and brow region, and general proportions became stronger identity anchors. hair, clothing, posture, body composition and aging were allowed to change.

that choice was not arbitrary. research into human face recognition repeatedly finds that the eye region carries particularly important identity information, while even removing eyebrows substantially damages recognition performance.[2][3]

i then created a dedicated library of character references and used qwen reference conditioning selectively for scenes where identity really mattered.

this exposed another useful limitation: stronger conditioning did not automatically produce a better image. it could preserve a face while pulling clothes, pose or composition toward the reference.

the solution was to use references like a scalpel rather than a default setting: stronger for one visible face, weaker for groups, and often absent entirely in wide or first-person shots.

the lock is gated by shot type: a reference drags composition with it, so it belongs only where a face actually shows.

aging without losing the character

following the same cast across decades introduced its own failure mode.

early generations aggressively interpreted age: characters in their thirties sometimes appeared grey, heavily lined and twenty years older than intended.

instead of simply adding more age words, i redesigned aging as a controlled progression. the goal was always:

the same person at another stage of life, not a newly generated older person.

perception research helped here too. facial contrast changes systematically with age and also affects how old a face is perceived to be.[4] it reinforced the idea that aging is carried by a combination of visual cues, not by turning “age” into a prompt synonym for wrinkles and grey hair.

structural identity remained stable while secondary cues were allowed to evolve gradually.

art-directing the camera

once the style and cast became consistent, another problem appeared. the images were starting to look too consistent.

an audit showed that almost half of the scenes relied on essentially the same framing.

instead of introducing random camera variety, i created one rule:

the camera should change because the feeling changes.

close framing could create intimacy or discomfort. wide frames could communicate loneliness or scale. first-person worked especially well when the scene was about physical sensation, fear or vulnerability.

this also borrowed from visual-narrative research: framing determines what receives attention inside a scene, while sequential images rely on the viewer to infer what happens between visible moments.[5][6]

each card therefore became less like three separate illustrations and more like a tiny visual sequence. i recomposed 182 situations using that approach.

explicit medium, medium-wide and two-person compositions rose from roughly 1.7% to 26.9%, while cards relying entirely on the same minimal framing fell from 46.8% to 20.1%.

the numbers were diagnostics, not quotas. emotion remained the art director.

Detail Masonry Slider

school

the years of being looked at

Detail Masonry Slider

leaving

the paperwork of becoming someone

Detail Masonry Slider

first work

rooms where decisions get signed

Detail Masonry Slider

mid-life

work, home, repetition

Detail Masonry Slider

elsewhere

a life conducted at arm’s length

Detail Masonry Slider

later years

slower rooms, kept things

Detail Masonry Slider

endings

night scenes and aftermath

Detail Masonry Slider

when the model got things wrong

a large visual audit found hundreds of images that needed correction or investigation.

the failures were wonderfully predictable: malformed hands, incorrect gaze, accidental text, duplicated people, strange clothing, unwanted objects and occasional anatomical inventions nobody had requested.

hands deserved particular attention. this is not just anecdotal frustration with image generators: recent research continues to treat anatomically correct hand synthesis as a distinct generation problem because articulation, occlusion and viewpoint remain difficult for general image models.[7][8]

different defects needed different responses. small failures were corrected locally. some scenes were tested with several edit strategies and compared manually.

but the most useful rule was knowing when not to edit. if the underlying generation was structurally wrong, repairing it usually created an increasingly fragile image. those frames were rejected and regenerated.

learning to edit less

another counterintuitive discovery came during retouching. longer edit instructions often produced worse results.

a detailed request gave the editing model permission to reinterpret unrelated parts of the frame: faces changed, lighting shifted, colors moved. short, surgical instructions preserved much more of the original image.

the rule became:

change the smallest possible thing required to solve the defect.

that same principle applied to the wider pipeline. more prompting was not always more control. more reference conditioning was not always more consistency. more editing was not always a better rescue.

route, don’t rely

there was never one model that made the art. there was a changing crowd of them, and the pipeline is essentially a routing decision: send each job to the model that wins that job, and keep a written record of the ones that lost.

both render routes run from the same manifest and the same seed, derived from the image id, so any frame is reproducible and the two routes stay comparable frame for frame. that also made the money predictable in advance: roughly six cents a frame, about seventy dollars for the deck, with a rework buffer on top and a standing rule that no paid batch runs without an explicit go-ahead.

around all of it turns one loop. it is genuinely cyclic — steps five through seven send work back to step one often enough that the log matters more than memory.

ai proposes; deterministic checks and a human dispose.

that sentence is the whole method. agents drafted, rewrote and reviewed at scale; a linter and one person decided. when an ai reviewer proved unreliable — over-flagging intended redesigns, occasionally inventing problems — it was demoted to advisory rather than trusted harder.

what did not work

some of the most useful decisions came from abandoning approaches that looked reasonable on paper.

what the project changed

the project began as an experiment in generating a large illustrated game. it became an exercise in controlling generative systems.

the biggest lessons were surprisingly simple:

and most importantly:

never confuse generation with art direction.

ai made producing 1,116 frames possible — and around a thousand more test frames behind them, spent on calibration and the search for the style. the real work was deciding what each frame should communicate, recognizing when the model missed it, understanding why, and building a reliable way to bring the result back under control.

selected research

1. 1hu, e. j. et al. lora: low-rank adaptation of large language models. iclr, 2022. arxiv:2106.09685.
2. 2sadr, j., jarudi, i. & sinha, p. the role of eyebrows in face recognition. perception 32(3), 2003. doi:10.1068/p5027.
3. 3royer, j. et al. greater reliance on the eye region predicts better face recognition ability. cognition 181, 2018. doi:10.1016/j.cognition.2018.08.004.
4. 4porcheron, a., mauger, e. & russell, r. aspects of facial contrast decrease with age and are cues for age perception. plos one 8(3), 2013. doi:10.1371/journal.pone.0057985.
5. 5cohn, n., taylor-weiner, a. & grossman, s. framing attention in japanese and american comics: cross-cultural differences in attentional structure. frontiers in psychology 3, 2012. doi:10.3389/fpsyg.2012.00349.
6. 6iyyer, m. et al. the amazing mysteries of the gutter: drawing inferences between panels in comic book narratives. arxiv:1611.05118, 2016.
7. 7zhang, h. et al. hand1000: generating realistic hands from text with only 1,000 images. arxiv:2408.15461, 2024.
8. 8chen, k. et al. foundhand: large-scale domain-specific learning for controllable hand image generation. arxiv:2412.02690, 2024.